“ Numerical Linear Algebra ” course

The main general reference for this and the next two lectures is a recent book [6] and its preliminary version [5] which is freely available on the web. 1 Why iterative methods? As we already know, there are two major classes of methods to solve linear systems: direct methods (like LU factorization, Cholesky factorization, etc.) and iterative methods. We underline the following properties of these groups of methods: direct methods: • do not preserve structure of the matrix; • hence, often are not suitable for large sparse matrices (as they introduce new non-zero entries, or the so-called fill-in); • typically have a robust " black-box " performance. iterative methods: • preserve structure of the matrix (in fact, the system matrix is often needed only for the matrix-vector products); • hence, are suitable for large sparse matrices (as they do not change the system matrix at all); • do not always have a " black-box " performance. In [4], the following estimate is given of the computational costs and memory costs required by direct and iterative methods for a representative class of model three-dimensional problems: methods computational costs memory direct O(n 2.3) O(n 1.7) iterative O(n 1.2) O(n) There is a large class of special direct methods that do try to preserve matrix sparsity as much as possible. These sparse direct methods have been significantly improved recently. Still, there are many problems for which iterative methods is the only possible way to solve the system.